Banghua/change resource server names - #291

Update resource server names to the following:

library_judge_math -> math_with_judge
multiverse_math_hard -> math_advanced_calculations
comp_coding -> code_gen
python_math_exec -> math_with_code
workbench -> workspace_assistant
multineedle -> example_multi_step

---------

Signed-off-by: banghuaz <banghuaz@nvidia.com>
Co-authored-by: Brian Yu <bxyu@nvidia.com>

Update resource server names to the following:

library_judge_math -> math_with_judge
multiverse_math_hard -> math_advanced_calculations
comp_coding -> code_gen
python_math_exec -> math_with_code
workbench -> workspace_assistant
multineedle -> example_multi_step

---------

Signed-off-by: banghuaz <banghuaz@nvidia.com>
Co-authored-by: Brian Yu <bxyu@nvidia.com>

Update resource server names to the following:

library_judge_math -> math_with_judge
multiverse_math_hard -> math_advanced_calculations
comp_coding -> code_gen
python_math_exec -> math_with_code
workbench -> workspace_assistant
multineedle -> example_multi_step

---------

Signed-off-by: banghuaz <banghuaz@nvidia.com>
Co-authored-by: Brian Yu <bxyu@nvidia.com>

…EADME (NVIDIA-NeMo#4574)

## What does this PR do?

The math_with_judge README's "Prepare for trajectory collection" command
works again. It loads `configs/dapo17k_trajectory_collection.yaml`,
which no longer exists, so `gym dataset collate` exited with
`ConfigPathNotFoundError` before downloading anything. This PR restores
that config in the current schema, documents the step's prerequisites,
and adds a regression test that replays the README command.

Running the README's other GitLab download, OpenMathReasoning, from
outside the Gym root exposed a second bug: the GitLab downloader did not
create the output directory. The second commit fixes it.

Fixes NVBug [6847517](https://nvbugspro.nvidia.com/bug/6847517).
Closes NVIDIA-NeMo#4498.

### Cause

NVIDIA-NeMo#859 ("Gym 0.2.0 huggingface dataset pointers") deleted three dapo17k
variants under `resources_servers/math_with_judge/configs/`, including
`dapo17k_trajectory_collection.yaml`, and removed their root-README
rows. The math_with_judge README, which has cited that file since NVIDIA-NeMo#291,
was not updated. Config resolution (`nemo_gym/global_config.py`
`load_extra_config_paths`) then raises `ConfigPathNotFoundError`.

`dapo17k.yaml` is not a drop-in replacement. The deleted config put the
DAPO17k train problems in the **validation** split, repeated 16 times,
because NeMo RL with `is_trajectory_collection: true` collects from the
validation dataloader only. `dapo17k.yaml` has DAPO17k as `train` ×1 and
AIME24 ×32 as `validation`, so trajectory collection would run on
AIME24.

Restoring the old file verbatim would not work either. It set
`judge_model_server.name: ???`, and since NVIDIA-NeMo#1575 the parser rejects any
`???` left after merging (`ConfigMissingValuesError`).

The second bug is in `download_jsonl_dataset`
(`nemo_gym/gitlab_utils.py`). It opened `output_fpath` for writing
without creating its directory. A Gym checkout tracks `data/`, so the
README's `--output data/open_math_reasoning_problems.jsonl` works from
the Gym root but fails with `FileNotFoundError` from any other
directory. `gym dataset collate --download` fails the same way for
GitLab-backed datasets. The Hugging Face download already creates the
directory (`hf_utils.py`).

### Fix

- Restore `configs/dapo17k_trajectory_collection.yaml`. The validation
split is the `dapo17k` 0.0.1 `dapo17k_train.jsonl` artifact (the same
GitLab source as `dapo17k.yaml`), `num_repeats: 16`. Grading uses
math-verify only (`should_use_judge: false`), so
`judge_model_server.name` is `policy_model` and the README command needs
no override. The file has `description`/`value` and the
environment-server block like `dapo17k.yaml`. The `update-readme-table`
hook re-adds its root-README row.
- It keeps its own local file,
`data/dapo17k_trajectory_collection_train.jsonl`, as before NVIDIA-NeMo#859.
Collate checks each dataset against `<jsonl stem>_metrics.json`. Sharing
`dapo17k.yaml`'s `dapo17k_train.jsonl` would compare the ×16 validation
stats with that file's train sidecar and fail (`train != validation`,
`17398 != 278368`).
- Add `data/dapo17k_trajectory_collection_train_metrics.json`, written
by a real collate run (278368 rows = 17398 × 16). Its word and turn
statistics match `dapo17k_train_metrics.json` exactly, so the rows are
the same.
- README: the command is unchanged. One note says the GitLab downloads
in this README (the OpenMathReasoning download and this `--download`)
need the `mlflow` extra (optional since NVIDIA-NeMo#3547) and `mlflow_tracking_uri`
/ `mlflow_tracking_token` in `env.yaml`. The section says what it writes
and why the split is validation.

Test:
`tests/unit_tests/test_math_with_judge_readme_trajectory_config.py`
takes the README's `gym dataset collate` block and runs it through the
real CLI with `dispatch` intercepted. It resolves the config offline (no
`env.yaml`), checks the single dataset is `validation` ×16 with
`dapo17k.yaml`'s GitLab identifier, and checks the committed sidecar
describes that split. It then runs `TrainDataProcessor` on a three-row
fixture in a temporary directory, without network access, and checks
that only `validation.jsonl` and its metrics come out, with each
question repeated 16 times in order.

- `download_jsonl_dataset` now creates the output's parent directory, as
the Hugging Face download does. A new test in
`tests/unit_tests/test_gitlab_utils.py` fakes the registry lookup and
the HTTP fetch, then downloads into a missing nested directory. It fails
before this change with the same `FileNotFoundError` and passes after.

These plausible wrong fixes all fail the trajectory test:
- restoring the file verbatim (`???`);
- pointing the README at `dapo17k.yaml` with a judge override;
- `num_repeats: 1`;
- a `train` split;
- sharing `dapo17k_train.jsonl`.

### Before the fix

Gym `977583d3d` (`main`), the README command from the Gym root:

```bash
gym dataset collate \
    --config resources_servers/math_with_judge/configs/dapo17k_trajectory_collection.yaml \
    --config responses_api_models/openai_model/configs/openai_model.yaml \
    --output-dir data/dapo17k_trajectory_collection \
    --mode train_preparation \
    --download
```

```text
nemo_gym.config_types.ConfigPathNotFoundError: config_paths entry 'resources_servers/math_with_judge/configs/dapo17k_trajectory_collection.yaml' was not found. Looked in:
  - <gym>/resources_servers/math_with_judge/configs/dapo17k_trajectory_collection.yaml
Check the path is spelled correctly and is relative to your working directory, an extra root
(NEMO_GYM_EXTRA_ROOTS / --search-dir), or the Gym install root.
EXIT=1
```

The README's OpenMathReasoning GitLab download, from a directory without
`data/`, same base:

```bash
gym dataset download --storage gitlab --name math_open_math_reasoning --revision 0.0.1 \
    --artifact open_math_reasoning_problems.jsonl --output data/open_math_reasoning_problems.jsonl
```

```text
  File ".../nemo_gym/gitlab_utils.py", line 111, in download_jsonl_dataset
    with open(config.output_fpath, "w") as f:
FileNotFoundError: [Errno 2] No such file or directory: '<workspace>/data/open_math_reasoning_problems.jsonl'
```

### After the fix

This branch, same command, with the `mlflow` extra and GitLab registry
credentials in `env.yaml`. No local data beforehand:

```text
Downloading dataset `validation` for `math_with_judge_simple_agent` from gitlab using dataset_name='dapo17k' version='0.0.1' artifact_fpath='dapo17k_train.jsonl' output_fpath='resources_servers/math_with_judge/data/dapo17k_trajectory_collection_train.jsonl'
Dataset validation repeating 16x: each line repeated 16 times (e.g. pattern: abc -> aaaabbbbcccc)
Aggregate metrics for resources_servers/math_with_judge/data/dapo17k_trajectory_collection_train_metrics.json
Aggregate metrics for data/dapo17k_trajectory_collection/validation_metrics.json
EXIT=0
$ wc -l data/dapo17k_trajectory_collection/validation.jsonl
278368 data/dapo17k_trajectory_collection/validation.jsonl
```

There are 17398 runs of identical questions, each 16 rows long, and no
`train.jsonl`. A second run skips the download, reports the sidecar
`already up to date`, and exits 0.

The OpenMathReasoning download, same command and directory: exit 0, with
the file written (the QA case below checks its rows).

### Validation

- QA testcase `math_with_judge`, which runs both README GitLab steps
from a scratch directory with registry credentials:
- base `977583d3d`: 0/2. Trajectory collation fails because the config
is missing; the download fails with `FileNotFoundError`.
- this branch: 2/2. The download passes in 4.4 s; the trajectory
collation passes in 20.3 s with 17398 × 16 rows and the committed
sidecar unchanged.
  - The QA-suite change ships separately.
- New tests: trajectory test before 1 failed
(`ConfigPathNotFoundError`), after 1 passed, and the five wrong fixes
above fail it. Downloader test before 1 failed (`FileNotFoundError`),
after 1 passed, also with the real `mlflow` 3.16.1 installed.
- `test_train_data_utils.py`, `test_cli_main.py`,
`test_global_config.py`, `test_dataset_source.py`,
`test_task_data.py::TestSchemaPresence`, `test_ray_opt_out.py`,
`test_cli_resources_servers.py`, `test_model_registry.py`,
`test_resources_server_registry.py`, `test_environment_scaffold.py`,
`test_gitlab_utils.py`, `test_hf_utils.py`, and the new tests: 592
passed.
- `pre-commit run --all-files`: passed.
- Rollout evidence: N/A. This is a config and docs change, and `gym
dataset collate` runs no model. The full download-and-collate run above
uses the real registry data.

### Compatibility

A new config file. Nothing that loads other configs changes. GitLab
downloads now create a missing output directory instead of failing. The
restored config uses the same GitLab artifact as `dapo17k.yaml`, but a
separate local file, so each keeps its own metrics sidecar.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: qiaochuz <qiaochuz@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

- Pin the version test to v0.7.0 (snapshots land before PyPI/NGC) and check
  backticked `vX.Y.Z` examples too.
- setup.mdx step 2: `ACCOUNT=<ACCOUNT_NAME>` is a Bash syntax error; use
  sample values like the multi-node page. The placeholder test now flags
  `VAR=<...>` assignments.
- FAQ dataset commands: multineedle was renamed to example_multi_step (#291);
  point at the existing config and example data, and define
  resource_config_path in the create-pr block. New test: dataset commands use
  a defined, existing resource config.
- vllm.mdx: the command sets --model-url, not policy_base_url=.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: qiaochuz <qiaochuz@nvidia.com>

…DIA-NeMo#4639)

## What does this PR do?

Three pages under `fern/versions/latest/pages` cite
`resources_servers/*/configs/*.yaml` files that do not exist on main.
This PR points each one at the file that replaced it. It also adds a
docs test so the next rename that leaves a citation behind fails CI.

No issue or NVBug: found by scanning the latest pages for cited config
paths and checking each one exists:

```bash
grep -rhoE 'resources_servers/[a-z0-9_]+/configs/[A-Za-z0-9_.-]+\.yaml' fern/versions/latest/pages \
  | sort -u | while read f; do [ -f "$f" ] || echo MISSING $f; done
```

| Page | Cited path | Now | Why |
| --- | --- | --- | --- |
| `reference/cli-commands.mdx` (`gym env prefetch` example) |
`resources_servers/math/configs/math.yaml` |
`resources_servers/math_with_judge/configs/math_with_judge.yaml` |
`resources_servers/math/` never existed. The example came from the
`prefetch` docstring added in NVIDIA-NeMo#2161 and was copied into the page by
NVIDIA-NeMo#4093. `math_with_judge.yaml` is the math config the same page already
uses for `ng_e2e_collect_rollouts`. The docstring in
`nemo_gym/cli/env.py` gets the same fix. |
|
`contribute/rl-framework-integration/gym-rl-framework-integration-success-criteria.mdx`
| `math_with_judge/configs/bytedtsinghua_dapo17k.yaml` (pinned to
`299e8c04`) | `math_with_judge/configs/dapo17k.yaml` on `main` | NVIDIA-NeMo#859
deleted `bytedtsinghua_dapo17k.yaml` and moved its DAPO17k train and
AIME24 validation sets into `dapo17k.yaml`. The pinned permalink still
resolves, but it points at a config that main no longer ships. The other
links in this page's tables already use `main`. |
| `reference/faq.mdx` (4 dataset upload/migrate examples) |
`resources_servers/multineedle/configs/multineedle.yaml` |
`resources_servers/example_multi_step/configs/example_multi_step.yaml` |
NVIDIA-NeMo#291 renamed `multineedle` to `example_multi_step`. The upload command
reads `domain` from this config, and `example_multi_step.yaml` sets it.
|

Versioned snapshots under `fern/versions/v*/` are frozen and left as
they shipped.

### Test


`tests/unit_tests/test_fern_docs_links.py::test_cited_resources_server_configs_exist`
scans every latest page for
`resources_servers/<name>/configs/<file>.yaml` and asserts that each
file exists. It skips `my_weather_tool`, which
`environment-tutorials/single-step-environment.mdx` has the reader
create. It also asserts that it found at least one citation, so a broken
regex can't make it pass vacuously.

## Before the fix

Gym `974d1a921` (main). The new test fails (`1 failed`) and reports
these six citations:

```
fern/versions/latest/pages/contribute/rl-framework-integration/gym-rl-framework-integration-success-criteria.mdx:38: resources_servers/math_with_judge/configs/bytedtsinghua_dapo17k.yaml
fern/versions/latest/pages/reference/cli-commands.mdx:695: resources_servers/math/configs/math.yaml
fern/versions/latest/pages/reference/faq.mdx:41: resources_servers/multineedle/configs/multineedle.yaml
fern/versions/latest/pages/reference/faq.mdx:53: resources_servers/multineedle/configs/multineedle.yaml
fern/versions/latest/pages/reference/faq.mdx:91: resources_servers/multineedle/configs/multineedle.yaml
fern/versions/latest/pages/reference/faq.mdx:108: resources_servers/multineedle/configs/multineedle.yaml
```

The documented `gym env prefetch` example, run with venv installs
stubbed out so that only config resolution and server discovery run
(`policy_model_name` is normally supplied by `env.yaml`):

```
$ gym env prefetch "+config_paths=[responses_api_models/local_vllm_model/configs/local_vllm_model.yaml,resources_servers/math/configs/math.yaml]" ++policy_model_name=Qwen/Qwen3-4B-Instruct-2507
Error: config_paths entry 'resources_servers/math/configs/math.yaml' was not found. Looked in:
  - <Gym>/resources_servers/math/configs/math.yaml
Check the path is spelled correctly and is relative to your working directory, an extra root
(NEMO_GYM_EXTRA_ROOTS / --search-dir), or the Gym install root.
exit=1
```

## After the fix

```
$ gym env prefetch "+config_paths=[responses_api_models/local_vllm_model/configs/local_vllm_model.yaml,resources_servers/math_with_judge/configs/math_with_judge.yaml]" ++policy_model_name=Qwen/Qwen3-4B-Instruct-2507
Pre-warming venv for: responses_api_models/local_vllm_model
Pre-warming venv for: resources_servers/math_with_judge
Pre-warming venv for: responses_api_agents/simple_agent
Pre-warming venv for: environment_servers/legacy_agent
Prefetch complete.
exit=0
```

`gym env resolve --config` now finds `dapo17k.yaml` and
`example_multi_step.yaml`. It then stops at the usual
unset-required-values check (`policy_model` / judge name), the same as
any config resolved without an `env.yaml`.

## Validation

- `pytest tests/unit_tests/test_fern_docs_links.py
tests/unit_tests/test_opensandbox_documented_config.py`: 25 passed, 21
subtests passed.
- `pytest tests/unit_tests/test_cli_main.py`: passed (the `env.py`
change only touches a docstring).
- `pre-commit run --all-files`: all hooks pass.
- Rollouts: N/A. The only change outside docs and tests is the example
text in one docstring.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: qiaochuz <qiaochuz@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

- Pin the version test to v0.7.0 (snapshots land before PyPI/NGC) and check
  backticked `vX.Y.Z` examples too.
- setup.mdx step 2: `ACCOUNT=<ACCOUNT_NAME>` is a Bash syntax error; use
  sample values like the multi-node page. The placeholder test now flags
  `VAR=<...>` assignments.
- FAQ dataset commands: multineedle was renamed to example_multi_step (#291);
  point at the existing config and example data, and define
  resource_config_path in the create-pr block. New test: dataset commands use
  a defined, existing resource config.
- vllm.mdx: the command sets --model-url, not policy_base_url=.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: qiaochuz <qiaochuz@nvidia.com>

…xamples to v0.7.0 (NVIDIA-NeMo#4579)

* docs: make r0.7.0 training commands copy-pasteable and bump version examples to v0.7.0

Training blocks export credentials first and reference $WANDB_API_KEY,
$HF_TOKEN, $SLURM_* and $USER instead of space-containing placeholders that
Bash splits into commands. Install, container, version-output and telemetry
examples move from v0.5.1 to v0.7.0 (published on PyPI and NGC; v0.6.0 has no
NGC image). Also fixes the FAQ dataset-name en dash and revision wording, slug
link labels, grammar, and British spellings. Adds regression tests.

Fixes NVBug 6875737 / NVIDIA-NeMo#4499

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: qiaochuz <qiaochuz@nvidia.com>

* docs: address review on r0.7.0 docs fix

- Pin the version test to v0.7.0 (snapshots land before PyPI/NGC) and check
  backticked `vX.Y.Z` examples too.
- setup.mdx step 2: `ACCOUNT=<ACCOUNT_NAME>` is a Bash syntax error; use
  sample values like the multi-node page. The placeholder test now flags
  `VAR=<...>` assignments.
- FAQ dataset commands: multineedle was renamed to example_multi_step (NVIDIA-NeMo#291);
  point at the existing config and example data, and define
  resource_config_path in the create-pr block. New test: dataset commands use
  a defined, existing resource config.
- vllm.mdx: the command sets --model-url, not policy_base_url=.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: qiaochuz <qiaochuz@nvidia.com>

* docs(tests): name the two placeholder shapes the bash-block test covers

Review note on NVIDIA-NeMo#4579: the test catches `{your ...}` placeholders and
`VAR=<...>` assignments, not every angle-bracket argument. Say so in the
docstring instead of claiming all placeholders bash splits.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: qiaochuz <qiaochuz@nvidia.com>

* docs(orchestration): label the prefill/decode link with its page title

The slug-label guard added in this PR flags `[how-to](...)` in
orchestration/configuration.mdx, a page that landed after this branch's
base (NVIDIA-NeMo#4760). Use the target page's title as the label.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: qiaochuz <qiaochuz@nvidia.com>

---------

Signed-off-by: qiaochuz <qiaochuz@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>